Semantic Video Search by Exploiting Large-Scale Visual Concepts

This paper summarizes our recent research works on semantic video search. Our solution to this exciting topic is grounded on concept-based video search, which attempts to narrow the semantic gap by first indexing the video content with a large number of pre-trained semantic concepts, and then employing these concepts to respond user’s queries (query-by-concept). We developed novel and effective techniques used in both stages, i.e., concept detection and semantic query-to-concept mapping. Our findings lead to several research prototypes, including VIREO-374 concept detectors [1] and query-to-concept mapping [4], which show state-of-the-art performance in TRECVID 2008 concept detection and automatic video search.